1. Field of the Invention
This invention relates generally to the field of computer software and computer program testing. More specifically, it relates to verification of graphics images produced by a computer graphics program.
2. Description of the Related Art
Early graphics systems were limited to two-dimensional (2D) graphics and acted as simple translators or interfaces to a display device. Modem graphics systems, however, must support three-dimensional (3D) graphics with textures and many special effects such as anti-aliasing, texture mapping, advanced shading, fogging, alpha-blending, and specular highlighting. Modem computer displays have many more pixels, greater color depth, and are able to display more complex images with higher refresh rates than earlier models. Consequently, modem graphics systems must incorporate graphics processors with a great deal of complexity, and the color value of one pixel can be the accumulated result of many calculations involving several models and mathematical approximations.
Software testing has experienced a similar transformation. As the complexity of the calculations and the layering of effects increased, the amount of time required to test and evaluate graphics software also increased and has become a significant bottleneck in the software development timeline. The evaluation of new graphics software may involve thousands of tests and require the judgment and experience of a senior test engineer to sort through the many possible reasons for a test failure. There may be more than one bug in a software program, and the cumulative result may cause a pixel in an output image to be different from a pixel in a reference image. In addition, there are many combinations and versions of hardware and software that may be used to generate a test image. It is not practical to generate a set of reference images for each of these combinations.
One method of testing graphics software is known as sample point testing. In this method the expected values of each pixel in a reference image generated by a specific hardware/software configuration is hardcoded in a test program. Each pixel of a test image is evaluated versus the hardcoded data in the test program. A difference image is formed by subtracting the value of a test image pixel from the value of the corresponding reference image pixel. Corresponding pixels are pixels located at the same x and y coordinates in both the test image and the reference image. Consequently, the test program user is easily overwhelmed, since the test evaluation output (the difference image) is as complex as the test image. In addition, the test program must be updated for any equipment or software modifications and each reference image requires a new test program. Maintaining and updating the test programs is both time consuming and error prone.
Other types of test programs are described in U.S. Pat. No. 6,226,406, issued on May 1, 2001, entitled “Method and Apparatus for Hybrid Sampling Image Verification” by Teh-Ming Hsieh.
A more efficient, faster, automated method of testing images generated on various combinations of computer graphics software and hardware using a limited set of reference images is needed. The method should accept insignificant rendering differences. The test result should be as simple as “pass or fail,” or provide a simple quantifiable measurement of the image comparison. The comparison criteria should be user adjustable and programmable.